SOSA: A Safe Load Shedding Approach for Monitoring Data Streams in Real-Time

Real-time stream processing is essential for many real-life stream-based applications. Systems designed to run such applications must be prepared to operate under overloaded conditions. Existing load shedding techniques are not suitable for processing data streams with stringent timing constraints because their tuple dropping policies may violate application deadlines in an uncontrolled way. To tackle this problem, we propose a Safe lOad Shedding Approach (SOSA) that aims to reduce the workload imposed on the system while at the same time preserve system timing constraints in this paper. By exploiting data stream semantics, SOSA categorizes stream processing into two different modes. This allows one to place provably lighter loads, under the (m, k) real-time scheduling model, on streams that operate in one particular mode. More importantly, we show that SOSA itself does not introduce any new timing failures. This property sets our work apart from other load shedding strategies for stream processing. To demonstrate the usefulness of SOSA, we introduce a concrete (m, k) scheduling algorithm called SOSA-DBP by combining SOSA with DBP, a well-known (m, k) scheduling algorithm. Probabilistic analysis and experimental results show that SOSADBP has significant performance gain over DBP.